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Response to Amendment 

1 . Amendments to claims have been entered and in view of the amendments 
new grounds of rejections have been introduced. 

2. New grounds of rejections have therefore been presented. 

3. Claims 1, 3-5 have been amended and Claim 2 is cancelled. 

4. Claims 6-17 are newly added. 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

Claims 1, 3-5 and 13 are rejected under 35 U.S.C. 103(a) as being unpatentable 

over Merchant (US 4635203) hereafter Merchant, in view of Hsu et al (6587601) 

hereafter Hsu. 

1. Regarding Claim 1, Merchant discloses a method (Figs 1,3a,3b and 4) for 
creating a novel viewpoint image from a plurality of images, the method 
comprising simultaneously acquiring sets of images from a plurality of cameras 
(as in Fig 3a,3b and Col 3 lines 25-33). Merchant further discloses the 
correlation of one image with the warped image to achieve a common image 
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which is a image-to-image focal plane transformed (novel viewpoint), geometrical 
differences (misregistration) at (Fig 4 and Col 1 Lines 46-67 through Col 2 
Lines 1-41 and Col 3 Lines 46-49). Merchant is silent and however does not 
however disclose using predetermined background based correspondence fields 
to detect novel objects; and assigning the image representations for these 
objects likely new correspondences; and using the resulting correspondences to, 
constructs a novel viewpoint image by warping at least one of the images. 
Wherein the novel viewpoint image corresponds to a view from a location 
different than a camera location. 

Hsu discloses using predetermined background based correspondence 
fields to detect novel objects (Figs 1-5, Col 1 Lines 52-55, Col 2 Lines 15-40, 
Col 3 Lines 55-56 and Col 4 lines 14-28); and assigning the image 
representations for these objects likely new correspondences (Col 5 lines 20- 
27); and using the resulting correspondences to, construct a novel viewpoint 
image by warping at least one of the images (Col 6 Lines 36-47, Col 13 Lines 
58-65),wherein the novel viewpoint image corresponds to a view from a location 
different than a camera location (synthetic reference images from the 
perspective of the camera). Hsu further discloses that a comprehensive view of 
the scene is produced at (Col 4 Lines 27-28). Hsu discloses that the reference 
image and the sensors images can be accurately aligned using the parametric 
transformation and the accuracy is up to a sub-pixel value at (Col 2 lines 5-32). 
Merchant and Hsu are from the same field of endeavor and are analogous art, 
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therefore it would have been obvious for one of ordinary skill in the art, at the 
time the invention was made to have used the teachings of Hsu in the method of 
merchant for the above reasons. 

2. Regarding Claim 3, Merchant and Hsu discloses the method in claim 1. Hsu 
further discloses wherein assignment of image representation likely new 
correspondences is random at (Col 5 Lines 20-30). 

3. Regarding Claim 4, Merchant and Hsu discloses the method in claim 1. Hsu 
further disclose comprising; testing the likely new correspondences and refining 
the likely new correspondences based on a result of the testing at (Col 13 lines 
58-65 and Col 7 Lines 58-61). 

4. Regarding Claim 5, Merchant and Hsu disclose the method in claim 1. Hsu 
further disclose wherein [[: a.]] parts of the scene that become visible in the 
novel viewpoint image for which no data are present in the image being warped 
because of occlusion in the said image being warped are provided by some other 
image from yet another viewpoint for which appropriate correspondences exist 
(Col 5 lines 20-30). 

5. Claim 1 3 is a corresponding system claim of the claim 1 . See the explanation 
of Claim 1. 
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Claims 6-10 are rejected under 35 U.S.C. 103 (a) as being unpatentable over 
Guo et al. (US 6353678) hereafter Guo in view of Hsu. 

6. Regarding Claim 6, Guo discloses a method (Figs 1-6) for efficiently 
generating a novel viewpoint image of a scene, the method comprising: acquiring 
a pair of background images of the scene (Fig 3 element 310); acquiring a pair 
of real-time images of the scene (Fig 3 Element 316). Guo also discloses the 
correspondences of the frames and warping of images to detect independent 
motions (movable objects) at (Fig 4, Col 8 Lines 36-54, Col 9 Lines 39-67 
through Col 10 Lines 1-21). Guo is silent and however does not disclose 
generating a background correspondence field based on the background images; 
detecting movable objects in the real-time images based at least in part on the 
background correspondence field; generating new correspondence data based at 
least in part on the movable objects. 

Hsu discloses generating a background correspondence field based on 
the background images (Figs 1-5, Col 1 Lines 52-55, Col 2 Lines 15-40, Col 3 
Lines 55-56 and Col 4 lines 14-28); detecting movable objects in the real-time 
images based at least in part on the background correspondence field; 
generating new correspondence data based at least in part on the movable 
objects (Col 5 lines 20-27). Hsu also discloses the warping of the images at (Col 
13 lines 57-58) and further discloses that the updated correspondence 
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information is used to generate new reference images to be used in the iterative 
matching at (Col 7 lines 58-61). Hsu further discloses that the reference image 
and the sensors images can be accurately aligned using the parametric 
transformation and the accuracy is up to a sub-pixel value at (Col 2 lines 5-32). 
Guo and Hsu are from the same field of endeavor and are analogous art, 
therefore it would have been obvious for one of ordinary skill in the art, at the 
time the invention was made to have used the teachings of Hsu in the method of 
Guo for the above reasons. 

7. Regarding Claim 9, Guo and Hsu disclose the method of claim 6. Hsu further 
disclose comprising integrating the new correspondence data into the 
background correspondence field by updating the parameters (correspondence 
data and field) to get the minimum error (SSD) at (Col 10 Lines 25-37). 

8. Regarding claim 10, Guo and Hsu disclose the method of claim 9. Hsu further 
disclose comprising repeating the steps of acquiring a pair of real-time images, 
detecting movable objects, generating new correspondence data, and integrating 
the new correspondence data in order to refine the background correspondence 
field at (Col 10 Lines 25-37). 

9. Regarding Claim 1 1 , Guo and Hsu disclose the method of claim 10. Hsu 
further disclose wherein the repeating step (iterative process) continues until a 
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difference (SSD) between the new correspondence data and the background 
correspondence field (when the optimum image alignment is achieved) is 
below a threshold (value of error parameter in equation 4 at Col 10 Lines 12- 
37). 



10. Regarding claim 12, Guo and Hsu disclose the method of claim 10. Hsu 
further disclose wherein the repeating step (iteratively processing) continues 
until a time limit expires (queries when the error minimization is complete at 
Col 10 Lines 12-24). 

Claims 7 and 8 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Guo in view of Hsu and in further view of Brandt et al. (US 6646655) hereafter 
Brandt. 



11. Regarding claim 7, Guo and Hsu disclose the method of claim 6. Guo and 
Hsu are silent and however do not disclose wherein the detecting step further 
comprises: warping a first of the real-time image pair into correspondence with a 
second of the real-time image pair; differencing the warped first real-time image 
and the second real-time image; and determining that difference values above a 
threshold correspond to the movable objects. 

Brandt discloses wherein the detecting step further comprises: warping a 
first of the real-time image pair into correspondence with a second of the real- 



Application/Control Number: 10/660,751 Page 8 

Art Unit: 2624 

time image pair; differencing the warped first real-time image and the second 
real-time image; and determining that difference values above a threshold 
correspond to the movable objects at (Col 6 Lines 13-38, Col 6 Lines 48-67, 
Col 7 Lines 36-67 through Col 8 Lines 1-6). Brandt discloses the Slide detector 
21, a frame comparator 34, and a camera motion detector 35 outputting a vector 
41 , foreground change detector 37 in Figs 3-5 which is important in detecting if 
the captured scene remains unchanged for numbers of frames or not at (Col 5 
Lines 10-28). This is important in detecting if there is a change in the foreground 
or not (movable object). Guo, Hsu and Brandt are from the same field of 
endevour and are analogous art, therefore it would have been obvious for one of 
ordinary skill in the art at the time the invention was made to have used the 
teachings Brandt in the method as disclosed by Guo and Hsu for the above 
reasons. 

12. Regarding Claim 8, Guo and Hsu disclose the method of claim 6. Guo and 
Hsu are silent and however do not disclose wherein the generating new 
correspondence data step further comprises: spatially grouping the detected 
movable objects; determining a distance between each spatial group and the 
background; and generating the new correspondence data based at least in part 
on the distance. 

Brandt discloses wherein the generating new correspondence data step 
further comprises: spatially grouping the detected movable objects (by a 
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bounding box around an area of change between successive frames at (Col 
6 Lines 23-24); determining a distance between each spatial group and the 
background and generating the new correspondence data based at least in part 
on the distance by a foreground change vector 43 at (Col 6 Line 22). 

Claims 14-17 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Merchant in view of Hsu and in further view of Brandt 

13. Regarding Claim 14, Merchant and Hsu disclose the system of claim 13. 
Merchant and Hsu are silent and however do not disclose wherein the processor 
is configured to detect the new objects by: warping a first of the subsequent 
images captured by a first camera into correspondence with a second of the 
subsequent images captured by a second camera; computing a difference 
between elements of the first image and corresponding elements of the second 
image; and determining that elements associated with difference values above a 
threshold correspond to new objects. 

Brandt discloses wherein the processor is configured to detect the new 
objects by: warping a first of the subsequent images captured by a first camera 
into correspondence with a second of the subsequent images captured by a 
second camera; computing a difference between elements of the first image and 
corresponding elements of the second image; and determining that elements 
associated with difference values above a threshold correspond to new objects at 
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(Col 6 Lines 13-38, Col 6 Lines 48-67, Col 7 Lines 36-67 through Col 8 Lines 
1-6). Brandt discloses the Slide detector 21, a frame comparator 34, and a 
camera motion detector 35 outputting a vector 41, foreground change detector 
37 in Figs 3-5 which is important in detecting if the captured scene remains 
unchanged for numbers of frames or not at (Col 5 Lines 10-28). This is important 
in detecting if there is a change in the foreground or not (movable object). 
Merchant, Hsu and Brandt are from the same field of endeavor and are 
analogous art therefore it would have been obvious for one of ordinary skill in the 
art at the time the invention was made to have used the teachings of Brandt in 
the system of Merchant and Hsu fro the reasons above. 

14. Regarding Claim 15, Merchant and Hsu disclose the system of claim 13. 
Merchant and Hsu are silent and however do not disclose wherein the processor 
is configured to generate new correspondence data by: spatially grouping the 
new objects; determining a distance between each new object spatial group and 
the background; and generating the new correspondence data based at least in 
part on the distance. 

Brandt discloses wherein the processor is configured to generate new 
correspondence data by: spatially grouping the new objects (by a bounding box 
around an area of change between successive frames at (Col 6 Lines 23- 
24); determining a distance between each new object spatial group and the 
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background; and generating the new correspondence data based at least in part 
on the distance by a foreground change vector 43 at (Col 6 Line 22). 

15. Regarding Claim 16, Merchant and Hsu disclose the system of claim 13, 
wherein the cameras are configured to continuously capture images and the 
processor is configured to continuously generate novel viewpoint images based 
on the continuously captured images (as in Fig 3a,3b and Col 3 lines 25-33 of 
merchant) and (Col 6 Lines 36-47 as in Hsu). 

16. Regarding Claim 17, Merchant, Hsu and Brandt disclose the system of claim 
16. Hsu further disclose comprising an output device (Fig 1 element 114) 
configured to output a video signal comprising the continuously generated novel 
viewpoint images. 

Conclusion 

Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 
37 CFR 1 .1 36(a). A shortened statutory period for reply to this final action is set 
to expire THREE MONTHS from the mailing date of this action. In the event a 
first reply is filed within TWO MONTHS of the mailing date of this final action and 
the advisory action is not mailed until after the end of the THREE-MONTH 
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shortened statutory period, then the shortened statutory period will expire on the 
date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1.136(a) will be calculated from the mailing date of the advisory action. In 
no event, however, will the statutory period for reply expire later than SIX 
MONTHS from the date of this final action. 

Any inquiry concerning this communication or earlier communications from 
the examiner should be directed to Jayesh A. Patel whose telephone number is 
571-270-1227. The examiner can normally be reached on M-F 7.00am to 4.30 
pm (5-4-9). If attempts to reach the examiner by telephone are unsuccessful, the 
examiner's supervisor, Jingge Wu can be reached on 571-272-7429. The fax 
phone number for the organization where this application or proceeding is 
assigned is 571 -273-8300. Information regarding the status of an application may 
be obtained from the Patent Application Information Retrieval (PAIR) system. 
Status information for published applications may be obtained from either Private 
PAIR or Public PAIR. Status information for unpublished applications is available 
through Private PAIR only. For more information about the PAIR system, see 
http://pair-direct.uspto.gov. Should you have questions on access to the Private 
PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 
(toll-free). If you would like assistance from a USPTO Customer Service 
Representative or access to the automated information system, call 800-786- 




